1. Field of the Invention
The present invention generally relates to data processing and more specifically to managing datastores.
2. Description of the Related Art
A management information system may contain one or more datastores to retain data related to various business functions. Such data may be critical to decision making, planning, program implementation, control etc. Furthermore, the ability to process, store and retrieve critical data quickly and efficiently may provide a business a competitive advantage in its market. For example, critical data may indicate shifting consumer expectations. By adjusting to such trends in the market, as indicated by previously collected and processed data, a business may become more profitable. In some instances the data collected may be critical not just to the future of the business itself but also to the safety of a current customer. For example, a hospital may maintain data related to a patient's medical records. The patient's current treatment may depend on past medical history to determine the safest solutions.
The use of an information system requires accesses to a datastore containing critical information to store or retrieve such information. However, a large number of accesses to the datastore may lead to serious degradations in performance of the information system. For example, large organizations may have hundreds of salesmen accessing the datastore to retrieve product and pricing information or to store information about recently made sales. Such large numbers of accesses to the datastore at the same time may severely strain the datastore which may have a limited bandwidth. As a result, data store accesses may become extremely slow and inefficient.
One solution to this problem is to maintain two datastores: a staging area datastore and an operational datastore. The staging area datastore may have information that has not yet been inserted into the operational datastore. For example, the staging area datastore of a hospital may contain data relating to the current clinical episode for a patient. A clinical episode may contain information relating to a particular visit to the hospital. The information relating to a current clinical episode, for example, may not be inserted in the operational datastore until that clinical episode is over. Because the data contained in the staging area may have a different format, such data may be normalized, annotated and checked for errors before insertion into the operational datastore. Insertions may be performed in batches during off peak hours when system time is available. This allows systems to be better utilized. As a result, users of the operational datastore have quicker response times and the system has a more consistent work load.
However, this solution may cause problems for users when the operational datastore is queried for recent information. Such recent information may not have been inserted in the operational datastore. Consequently, the result sets returned for queries against the operational data store may contain incomplete or even in accurate data. One solution for this problem is to write an application that reads from both the operational datastore and the staging area. However, because the data contained in the operational data store and the staging datastore may be in different formats, two applications will have to be written to represent the same type of data. Additionally, as previously described, the data in the staging area may not be normalized, annotated etc., in the same way as data in the operational data store, thereby limiting the way in which it can be searched.
Therefore, what is needed is improved methods and systems to provide meaningful results of a query against a first datastore by including relevant information contained in the second datastore.